Count-Min Sketch
نویسنده
چکیده
DEFINITION The Count-Min (CM) Sketch is a compact summary data structure capable of representing a high-dimensional vector and answering queries on this vector, in particular point queries and dot product queries, with strong accuracy guarantees. Such queries are at the core of many computations, so the structure can be used in order to answer a variety of other queries, such as frequent items (heavy hitters), quantile finding, join size estimation, and more. Since the data structure can easily process updates in the form of additions or subtractions to dimensions of the vector (which may correspond to insertions or deletions, or other transactions), it is capable of working over streams of updates, at high rates. The data structure maintains the linear projection of the vector with a number of other random vectors. These vectors are defined implicitly by simple hash functions. Increasing the range of the hash functions increases the accuracy of the summary, and increasing the number of hash functions decreases the probability of a bad estimate. These tradeoffs are quantified precisely below. Because of this linearity, CM sketches can be scaled, added and subtracted, to produce summaries of the corresponding scaled and combined vectors.
منابع مشابه
Count-Min-Log sketch: Approximately counting with approximate counters
Count-Min Sketch [1] is a widely adopted algorithm for approximate event counting in large scale processing. However, the original version of the Count-Min-Sketch (CMS) suffers of some deficiences, especially if one is interested in the low-frequency items, such as in textmining related tasks. Several variants of CMS [5] have been proposed to compensate for the high relative error for low-frequ...
متن کاملCount-Min Sketches for Estimating Password Frequency within Hamming Distance Two
The count-min sketch is a useful data structure for recording and estimating the frequency of string occurrences, such as passwords, in sub-linear space with high accuracy. However, it cannot be used to draw conclusions on groups of strings that are similar, for example close in Hamming distance. This paper introduces a variant of the count-min sketch which allows for estimating counts within a...
متن کاملAn Improved Data Stream Summary: The Count-Min Sketch and Its Applications
We introduce a new sublinear space data structure—the count-min sketch—for summarizing data streams. Our sketch allows fundamental queries in data stream summarization such as point, range, and inner product queries to be approximately answered very quickly; in addition, it can be applied to solve several important problems in data streams such as finding quantiles, frequent items, etc. The tim...
متن کاملAlgorithmic Techniques for Big Data
Handling an Update: When an update (it, ct) arrives, then ct is added to one entry in each row of the array count. Specifically, ∀1 ≤ j ≤ d, count [j, hj(it)]← count [j, hj(it)] + ct. Lemma 1. The space used by Count-Min Sketch is O(wd) ≡ O( ln 1 δ ) words. Specifically, it uses an array which takes wd words and d hash functions, each of which can be stored using 2 words. An update can be handl...
متن کاملLossy Conservative Update (LCU) Sketch: Succinct Approximate Count Storage
In this paper, we propose a variant of the conservativeupdate Count-Min sketch to further reduce the overestimation error incurred. Inspired by ideas from lossy counting, we divide a stream of items into multiple windows, and decrement certain counts in the sketch at window boundaries. We refer to this approach as a lossy conservative update (LCU). The reduction in overestimation error of count...
متن کامل